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D : Abstract 

A search for the lightest neutral scalar and neutral pseudoscalar Higgs bosons 
in the Minimal Supersymmetric Standard Model is performed using 176.4 pb _1 
of integrated luminosity collected by L3 at a center-of-mass energy of 189 GeV. 
No signal is observed, and the data are consistent with the expected Standard 
Model background. Lower limits on the masses of the lightest neutral scalar and 
pseudoscalar Higgs bosons are given as a function of tan/3. Lower mass limits for 
tan/3 > 1 are set at the 95% confidence level to be m h > 77.1 GeV and m^ > 
77.1 GeV. 
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Introduction 

In the Minimal Supersymmetric Standard Model (MSSM) [1 two Higgs doublets are required, 
giving rise to five Higgs bosons: a charged scalar pair, H ; two neutral scalars, h and H; and a 
neutral pseudoscalar, A. Within this framework, the h and A are predicted to be the lightest 
Higgs particles and, therefore, the most likely to be observed at LEP. The two main production 
mechanisms are investigated in this letter: 



e + e"^Z*^hZ (1) 

e + e"->Z*^hA. (2) 

Process (JIJ is very similar to the dominant Standard Model Higgs production mechanism, 
for which L3 has set a lower limit on the mass of the Higgs at 95.3 GeV (2j- The production rate 
for (£Q) is, in general, smaller than that of the Standard Model reaction, but this is compensated 
by the additional pair-production process (J2J). 

Previous searches for the h and A bosons have been reported by L3 [3] and other exper- 
iments jl]. In this letter, our sensitivity to these particles is extended by including the data 
taken at \/s = 189 GeV and by scanning over a larger MSSM parameter space. 

Data and Monte Carlo Samples 

The data were collected using the L3 detector [5 at LEP during 1998. The integrated luminosity 
is 176.4 pb _1 at an average center-of-mass energy of 188.7 GeV. 

The signal cross sections and branching ratios are calculated using the HZHA generator [S|. 
For the efficiency studies, Monte Carlo samples of Higgs events are generated using PYTHIA j^j 
and HZHA. For the background studies, the following Monte Carlo programs are used: PYTHIA 
(e+e-->qq(7)), KORALW |gj (e+e" -> W + W~), KORALZ |j (e+e-^r+r-), PHOJET [E] 
(e+e-->e + e-qq), EXCALIBUR [HI (e + e" -»■ ff'f'f") and PYTHIA (e+e" -»■ ZZ and e+e" -> 
Ze + e~). The number of simulated background events for the most important background 
channels is typically 100 times the number of collected data events. The Monte Carlo signals 
are 300 times the number of events expected to be observed with these luminosities. 

The L3 detector response is simulated using the GEANT 3.15 program [12], which takes 
into account the effects of energy loss, multiple scattering and showering in the detector. The 
GHEISHA program ^3] is used to simulate hadronic interactions in the detector. 

Analysis Procedures 

The search for hA and hZ production is carried out within a constrained MSSM assuming 
unification of the scalar fermion masses, the gaugino masses and the trilinear Higgs-fermion 
couplings at the GUT scale. This choice has little impact on the Higgs mass phenomenology 
but reduces significantly the number of free parameters. The universal scalar fermion mass m 
and the gaugino mass parameter M 2 are fixed to 1 TeV. The Higgs mass parameter \x is set 
to —0.1 TeV. Two extreme scenarios are considered corresponding to maximal and minimal 
scalar top mixing as suggested in Reference [Hj. The minimal mixing scenario corresponds 
to setting the trilinear Higgs-fermion coupling A to zero. Maximal scalar top mixing occurs 
at A = v^6 TeV. A scan is then performed, in each mixing scheme, over the two remaining 



free parameters m\ and tan/?. For this search, the minimum value of tan/3 considered has been 
decreased from 1.0 to 0.7 and the minimum A mass considered has been decreased from 30 GeV 
to 10 GeV with respect to our previous publication. Values of m\ in the range rriA < 10 GeV 
have been previously excluded at LEP |15j . 

The two Higgs production mechanisms, e + e~ — > Z* — > hA and e + e _ — > Z* — > hZ, vary in 
relative importance as a function of tan/3. The production of hA is dominant at high tan/3, while 
hZ production is dominant at low tan/3. The description of the hZ analyses at y/s = 189 GeV 
of the decay modes other than hZ — »bbqq and hZ -^bbr + r _ can be found in Reference [2j. The 
analyses for hZ— >bbqq and hZ^bbr + r _ (t + t~ qq) used in this letter have been optimized to 
account for the analogous signatures in the hA channel: hA— >bbbb and hA— >bbr + r _ . 

For values of m^ less than 30 GeV, decays of the h into a pair of A bosons become possible. 
The A decays predominantly to b quarks and tau leptons for most of the tan/? region probed. 
The hZ — > bbqq analysis has a significant cross-efficiency for the hZ — > AAf f channel and is used 
to search for this process. 

Common search procedures are applied to both the hA and hZ channels. First, a preselection 
is applied which significantly reduces background while keeping high signal efficiency This is 
especially effective against background from the two-photon interaction, which has a large cross 
section at these LEP energies. Second, a final set of selection cuts is chosen to distinguish signal 
from background. Once the final selection has been applied, a discriminating variable as defined 
in Reference 0E] is calculated for each scan point in the (tan/?, m&) plane. 

There is a significant overlap in the selection for hA and hZ in both the channels involving 
either four jets, or two jets and two taus. The confidence level calculation requires that all 
events be uniquely assigned to a given channel. To this end, for events that pass both the hA 
and hZ selections, an unique assignment is made based on the reconstructed masses and the 
relative production rates at each scan point. 

The hA — >bbbb and hZ — ►bbqq Channels 

The signature of both the hA— >bbbb and hZ— >bbqq decay modes is four high- multiplicity 
hadronic jets and the presence of b hadrons. The dominant backgrounds come from qq produc- 
tion and hadronic decays of W pairs and Z pairs. In the case of hA— >bbbb, the identification of 
b hadrons plays an especially important role. The analysis follows closely that of Reference [Sj. 

First, a high multiplicity hadronic preselection, common to both hA and hZ, is applied 
which eliminates background from the two-photon interaction. The preselection is similar to 
the one used at y/s = 183 GeV and only minor changes are made to account for the increased 
center-of-mass energy. Events passing the preselection are then forced to have four jets using the 
DURHAM [16; clustering algorithm, and a kinematic fit requiring four-momentum conservation 
(4C) is performed. 

Once the preselection has been satisfied, an optimization procedure is applied on the Monte 
Carlo to choose cuts on variables that maximize the separation between signal and background. 
These optimized cuts serve mainly to reject the multi-jet QCD background and are dependent 
on the topology being investigated: hA or hZ. Selection cuts are placed on the maximum and 
minimum dijet mass, minimum jet energy, maximum jet energy difference and on Y34, being 
the value of the DURHAM jet resolution parameter for which the event is transformed from 
a four-jet to a three-jet topology. Values of the cuts for the hA and hZ analyses are shown in 
Table d The number of observed and expected events from Standard Model processes in the 
y/s = 189 GeV data along with the signal efficiencies for the preselection and selection cuts are 



shown in Table El 

Events passing the selection cuts are then classified in three categories: 1) those that pass 
only the hA cuts; 2) those that pass only the hZ cuts; and 3) those that pass both sets of cuts. 
Category 3) is then split into two separate samples by choosing the most likely hypothesis based 
on the relative production rate for hA and hZ and the probability of the mass y 2 as defined in 
Reference jHj • 

In the final step, the analysis is optimized for four regions in the (tan/3, rrih) plane near the 
limit of our discovery potential. For this, the B ia% variable (Figure la), the Higgs production 
angle with respect to the beam axis, 0, (Figure lb) and the probability for the \ 2 of the Higgs 
mass hypothesis (Figure lc) are used. The relative discriminating power of these variables 
changes with the Higgs mass hypothesis. For this reason, a cut optimization is performed at 
four points in the (tan/3, m h ) plane: (2.7,95 GeV), (7.5,80 GeV), (20,80 GeV) and (50,80 GeV). 

The final discriminating variable is the logarithm of the weighted combination of the prob- 
abilities of the -B t ag and the mass \ 2 to be consistent with background. Distributions of the 
final discriminant for the hA search and the hZ search are shown in Figure El 

The hZ^AAff Channel 

To investigate h decays into A-pairs in the region of very low tan/3 and low m\, where this 
channel becomes dominant, the hZ four jet analysis described above is employed. The signature 
of this process is at least four hadronic jets with very high probabilities to contain b quarks. The 
preselection and optimized cuts chosen for the four jet analysis are applied without adjustment. 
The efficiency on hZ — > AAZ ^bbbbqq is above 40% over the region of interest. The mass \ 2 
of the four jet analysis is less effective in the six jet topology, however the B ta , g gives the final 
variable enough discriminating power to distinguish between signal and background. 

The hA— >bbT + r - , hZ— >bbr + r - and hZ— >r + T _ qq Channels 

The signatures of hA— >bbr + r~, hZ— >-bbr + r~ or hZ^r + r~qq evento are a pair of taus 
accompanied by two hadronic jets. The main background comes from W-pair decays containing 
taus. Two analyses are optimized for the hZ and for the hA channels. The hZ analysis follows 
that of the Standard Model Higgs search and is described in detail in Reference J2j. The 
hA— »-bbr + r _ selection is described in this letter. As in the Standard Model Higgs search, two 
selections are performed, one based on the tau identification (particle-based selection) and the 
other relying more on the event kinematics (jet-based selection). 

First a common preselection is applied to both analyses, then cuts specific to each analysis 
are chosen. The major difference in the hA selection from that of the hZ analysis is the need 
for greater sensitivity to lower Higgs masses. To accomplish this, the cuts on opening angles 
of the jet and tau pairs have been removed, and the invariant mass cuts on the tau-tau and 
jet-jet systems have been relaxed. To reject the increased background accepted by loosening 
the selection, additional cuts are applied which exploit the kinematics of the hA events. A cut 
is placed on the ratio of the sum of the energies of the tau decay products to the sum of the 
jet energies. The magnitude of the missing momentum vector in the rest frame of the Higgs is 
restricted, where the taus are expected to be back-to-back resulting in a partial cancellation of 
the missing momentum vectors. Finally, there is a requirement on the cosine of the production 
angle of the Higgs boson with respect to the beam axis similar to that in the four-jet hA 

^The hA^r + r _ bb is also considered. 



analysis. The selection cuts chosen for both the particle- and jet-based selections are shown in 
Table El The number of events observed, the number expected from background processes, and 
the signal efficiency for the hA and hZ analyses, after combining the particle- and jet-based 
selections, are shown in Table EJ 

The final variable is the likelihood of the event to be hA or hZ based on the B tag values for 
each hadronic jet, shown in Figures EH andEb, and the reconstructed invariant mass of either 
the jet or tau system, shown in Figures Et and Eli, using the same technique as in the Standard 
Model Higgs search. Events which pass the hA as well as the hZ selection are classified as either 
hA or hZ depending on the cross section weighted values of these likelihoods. Examples of the 
final variable for the hA search at large values of tan/3 and the hZ search at low values of tan/3 
are shown in Figure 01 

Results 

No evidence of the production of the h and A bosons is observed in the data. The excluded 
region of the MSSM parameter space is evaluated by calculating the confidence level (CL) that 
the expected signal is absent in the observed data for the plane defined by (tan/3, tha)- The 
CL is calculated using the technique described in References jTHHH] . Bins of an analysis with 
a signal-over-background ratio in the Monte Carlo of less than 0.05 are not considered in the 
calculation of CL. This cut is chosen to minimize the effect of systematic errors on the average 
CL as calculated from a large set of Monte Carlo trials. 

Systematic errors on the signal and background are considered using the same procedure 
as in the Standard Model Higgs searches [21E1EI]- The overall systematic error is estimated 
to be 5% on the number of signal and 10% on the number of background events. Statistical 
uncertainties due to Monte Carlo statistics are completely uncorrelated among the different 
bins of the individual channels and have little effect on the final CL calculation. 

The data from the MSSM Higgs search using lower center-of-mass energies J2] is combined 
with the y/s = 189 GeV data. Figure El shows the region of the (tan/3, m^) plane and the 
(tan/3, tua) plane excluded by L3 for the maximal and minimal mixing scenarios. On the plot, 
the 95% CL is shown as a solid line while the expected median CL is shown as a dashed 
line. Table E] lists the masses of the h and A excluded at the 95% CL using the data at 
■y/s = 189 GeV and lower center-of-mass energies for tan/3 = 3 and tan/3 = 50 as well as the 
median and average exclusion and the probability to obtain a higher limit. The probability 
to obtain a higher limit reaches a maximum in the high tan/3 region with an m^ mass of 
68 GeV, where there is an upward fluctuation in the data. The lowest value of m^ excluded is 
at tan/3 = 15.0 for maximal mixing and the lowest value of itia is excluded at tan/3 = 50.0 for 
minimal mixing. An interesting feature of these results is that the region of 0.8 < tan/3 < 1.5 is 
excluded in the MSSM, according to the current theoretical calculation of the maximum Higgs 
mass allowed and for m top equal to 175 GeV [Ej. However, recent two- loop calculations |2HJ 
seem to favor larger values of the maximum allowed m^ in this region, which would change the 
excluded band of tan/3. 

For the MSSM parameters considered and assuming tan/3 greater than one, this results in 
lower mass limits at the 95% CL of 



m h > 77.1 GeV, m A > 77.1 GeV. 
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Cut 


hA 


hZ 


Minimum dijet mass (GeV) 


> 15.7 


> 25.3 


Maximum dijet mass (GeV) 


< 135.3 


< 118.7 


Minimum jet energy (GeV) 


> 15.1 


> 25.9 


Maximum A£j et (GeV) 


< 54.8 


< 42.4 


yD 


> 0.003 


> 0.009 


Visible Energy (GeV) 


> 129.3 


> 133.8 


Number of Tracks 


> 25 


> 22 



Table 1: Selection cuts for the hA and hZ four-jet Higgs search channels. In addition to those 
abbreviations defined in the text, the symbol A£j et is the energy difference between any two 
jets of the four-jet system. 





Numbi 


sr of Event 


,s 


Process 


Preselection 


hA cuts 


hZ cuts 


e + e~ — >e + e~qq 


7.1 


0.7 


0.6 


e + e~ — >qq 


758.0 


203.7 


57.4 


e+e-^W+W" 


1331.7 


913.5 


582.1 


e+e-^ZZ 


76.0 


47.5 


37.6 


Total Expected 


2172.8 


1165.4 


677.7 


Data 


2141 


1110 


641 


Efficiency hA^bbbb 


91.5% 


77.1% 


43.6% 


Efficiency hZ — > bbqq 


93.3% 


78.2% 


66.2% 



Table 2: Number of events expected and observed in the four-jets channels. The signal efficien- 
cies at y/s = 189 GeV are quoted for hA at m^ = ttl^ = 80 GeV and for hZ at m^ = 95 GcV. 



Cut 


Particle-based selection 


Jet-based selection 


Number of tracks 


> 5 


> 5 


Number of clusters 


> 15 


> 15 


-EWv^ 


> 0.4, < 0.95 


> 0.4, < 0.90 


-^- / e)-C / ^t)-^ / 7 


< 40 GeV 


< 40 GeV 


1111 34 


> -6 


> -6 


E T 


< 1 


< 1 


fn TT ,TTlqq 


> 5 GeV,< 125 GeV 


> 5 GeV,< 125 GeV 


COS 


< 0.8 


< 0.8 


\ V* ■ 
1 i^miss 1 


< 40 GeV 


< 40 GeV 


I COS^f m i ss J | 


- 


< 0.95 


"r-jet 


- 


> 25° 



Table 3: Selection cuts for particle-based and jet-based tau selections in the hA^bbr + T~ 
search channel. In addition to those abbreviations defined in the text: E vis is the visible energy; 
E e , E^ and E 1 are the electron, muon and photon energies, respectively; E T is the ratio of the 
sum of the energies of the tau decay products to the sum of the jet energies; m TT ,m qq is the 
invariant mass of the tau-tau and jet-jet systems, respectively; G is the production angle of the 
Higgs boson with respect to the beam axis; p miss is the magnitude of the missing momentum 
vector in the rest frame of the Higgs; G m i SS is the angle of missing energy vector with respect 
to the beam axis; and a r _j Ct is the angle between a tau jet and the closest quark jet. 







Number of Events 


Process 




hA selection 


hZ selection 


e + e~— >qq 




2.3 


2.3 


e+e-^W+W" 




11.3 


11.2 


e+e-^ZZ 




2.6 


3.1 


e + e~ — > Ze + e~ 




0.4 


0.5 


Total Expected 




16.6 


17.1 


Data 




20 


12 


Efficiency hA— >bbr~ 


f T" 


35.2% 


35.4% 


Efficiency hZ^bbr + r _ 


21.1% 


30.0% 


Efficiency hZ ^t + t~ 


•qq 


21.8% 


29.8% 



Table 4: Number of events expected and observed after selection for the tau search channels. 
The signal efficiencies at yt~s = 189 GeV are quoted for hA at itla = irih = 80 GeV and for hZ 
at mi = 95 GeV. 





Lower mass limits 


in GeV at 95% CL 






Observed 




Expected 






Mixing, tan/5 


m h m A 


<7«h> 


<mA> mh 


m A 


CL b 


minimal, 3 


96.3 225.0 


92.7 


164.0 94.6 


192.6 


12% 


minimal, 50 


77.1 77.1 


78.2 


78.2 80.0 


80.0 


80% 


maximal, 3 


95.4 128.9 


89.0 


111.9 90.4 


117.1 


15% 


maximal, 50 


77.5 77.6 


78.9 


79.0 81.4 


81.5 


77% 



Table 5: Higgs mass limits in the MSSM from the data at ^/s = 130 GeV — 189 GeV. The 
masses in boldface are the lower mass limits set at the 95% CL from the data. The masses <m> 
and m are respectively the average and median mass limits for the h and A bosons as calculated 
from a large set of Monte Carlo trials. Assuming there is no signal, CL^ is the probability to 
obtain a mass limit on m^ larger than the one observed. 
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Figure 1: Distributions of the a) B tag and b) cosine of the Higgs production angle in the 
four-jets search. The hatched histogram is the expected hA signal (multiplied by a factor of 
50) for mh = 80 GeV and tan/5 = 50. Distribution c) is the logarithm of the probability of the 
mass x 2 - The hatched histogram is the expected hZ signal (multiplied by a factor of 10) for 
mi = 95 GeV and tan/3 = 3. 
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Figure 2: Distributions of the final discriminant for the category of events passing a) both 
hA and hZ cuts but classified as hA candidates and b) events passing only the set of cuts for 
hA. The hatched histogram is the hA signal expectation for m^ = 80 GeV and tan/5 = 50. 
Distributions are plotted for c) the events passing both hA and hZ cuts but classified as hZ and 
d) events passing only the hZ selection. The hatched histogram is the hZ signal expectation 
for rrih = 95 GeV and tan/3 = 3. 
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Figure 3: The distributions for the hA— >bbr + r search channel of a) the B ta , g for hadronic 
jet 1 and b) hadronic jet 2, c) the reconstructed mass for the hadronic system, and d) the 
reconstructed mass for the tau-tau system. The hatched histogram is the hA— >bbr + T~ signal 
normalized for mj, = 80 GeV and tan/3 = 50. 
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Figure 4: Distributions of the final variables for the a) hA^bbr + T~ for m^ = 80 GeV at 
tan/5 = 50, b) the hZ— >bbr + T~ for mi, = 95 GeV at tan/5 = 3 and c) the hZ— >T + r~qq search 
for mh = 95 GeV at tan/5 = 3. 
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Figure 5: Exclusion plots of the Higgs mass versus tan/3 at the 95% CL. In all plots the area 
shaded by diagonal lines is the 95% exclusion, while the cross-hatched region is theoretically 
disallowed. The grey region in plots a) and c) corresponds to m^ < 10 GeV and has been 
previous excluded at LEP [T3]. Plot a) is the 95% CL exclusion of m^ versus tan/3 in the 
minimal mixing scenario, and b) is the 95% exclusion of rriA versus tan/3 also for minimal 
mixing. Plots c) and d) are the same for the maximal mixing scenario. 



